Skip to content

docs(spec): specify the sheet content type (S1 design) - #236

Open
rabbitson87 wants to merge 2 commits into
mainfrom
docs/sheet-s1-design
Open

rabbitson87 wants to merge 2 commits into
mainfrom
docs/sheet-s1-design

Conversation

@rabbitson87

@rabbitson87 rabbitson87 commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds the S1 design of the sheet content type from issue #233, as two spec pages and one index row in each of the two indexes:

  • docs/spec/sheet.md: the product spec. A workbook of worksheets that people and agents edit live, formulas that follow Excel, xlsx and CSV import and export, revisions, agent tools, Public read-only access. Status: planned.
  • docs/spec/sheet-technical-design.md: the technical design. Storage, formulas and the dirty mark, calculation, jobs, the xlsx reader and writer, the live protocol, the agent interface, limits, validation plan and open technical questions. Status: planned.

This is the design stage only. The change adds no code, no guide topic, no tool and no generated reference. Nothing is implemented, and nothing here is released.

Refs #233. It does not close the issue.

What the design decides

  • A host content type on the Revision model. No worker, no build step, and it runs with the server runtime off.
  • Formulas calculate in the host, in Go. One implementation of every function, of input parsing and of number formats.
  • Rows and columns have stable ids and a separate order. A formula stores ids, so an insert rewrites no formula.
  • Results are a cache. An edit writes small records, and a marker turns them into marks in bounded slices. While marking is open, a reader recomputes from inputs or shows pending. Two bounds (10,000 edits, 60 seconds) keep that state from lasting for ever.
  • The xlsx reader and writer are built on archive/zip, with no new module. Import, export, rollback and large duplication are jobs with progress and cancel.
  • 111 built-in functions, merged cells and names, typed input in year-first, Korean and US-order forms, and a conformance corpus that the requester saves once in Excel.
  • Scale: 1,048,576 rows and 16,384 columns for a worksheet, 20,000,000 cells for a workbook.
  • A large sort. A sort of 100,001 to 2,000,000 cells runs as a job that plans while people keep editing and applies in one writer item. The apply pauses writes to the sheet for seconds. It ends as conflict if a sort key or the structure changed after the plan.
  • Whole-column references cover the whole grid, as in Excel: ROWS(A:A) is 1,048,576.

Decisions of the requester

The requester read the effect of each open question on 2026-11-14 and decided them. Questions 3 to 13 and 15 to 17 of the spec stand as the design wrote them. The requester changed two:

  • Question 14: a sort may pass the cell limit of one action, through the large sort job. A sort of more than 2,000,000 cells is still refused, so a sort of large workbook B (about 20,000,000 cells) is not possible.
  • Question 16: a whole-column reference covers the whole grid. This costs only the rows that exist for a function that reads the column directly. An array expression over a whole column, such as SUMPRODUCT(--(A:A="")), holds all 1,048,576 positions and costs their time, as in Excel. A lazy array that skips the empty positions would remove that cost, and it is a new mechanism that the design does not have. The requester has not decided it.

Two consequences need the requester's attention:

  • A formula that gives #LIMIT! from an array over whole columns, or from an array of more than 4,194,304 values, is left out of the formula conformance criterion.
  • The apply of a large sort is the one foreground item that holds the writer for longer than a unit. It takes about 10 seconds for 2,000,000 cells without formulas and about 40 seconds with formulas, by estimate.

Evidence

  • The S0 prototype was a server flat outside the repository. Its three reports are on feat(sheet): add a sheet content type with xlsx import and export, formulas and notebook links #233 and give the storage, read, edit, insert and mark numbers.
  • Small probes outside the repository, in Go 1.27.1 with the SQLite of go.mod, measured: the eager dirty mark (3.98 s for 1,000,000 dependents), a marker in slices (slice p95 17 to 19 ms), xlsx read (8.8 s for 18.7 million cells with a minimal scanner, 62.6 s with excelize) and write (4.2 s and 18 MiB for 20 million cells), and FTS5 (works in the bundled SQLite; 22.9 bytes for each distinct string with detail=none).
  • Not measured: a chain of 1,000,000 links, a two-level fan-out, the pending-mode recalculation, the 60 second bound under a real stream of edits, the stab past 2,000 ranges, the largest window, the speed of a full-featured xlsx reader, and the apply time, key read and digest of the large sort. Section 26 of the technical design lists each one. Excel was not run, so open question 12 and the conformance corpus wait for the requester's Excel files.

Review record

Stage: design, standalone. Candidate verdict: accepted by reviewers of a different model, as described below. Implementation and testing are not assessed. Publication is this PR. Merging is not authorized by this description.

  • Authors: claude-fable-5-2 (first documents and probes) and claude-sonnet-5-5 (all corrections).
  • Reviewers, in fresh sessions, with the effective model from the execution record: claude-opus-5-5 (reviewer A) and claude-haiku-5-5 (reviewer B).
  • Rounds: twelve snapshots. Rounds 1 to 4 reviewed the whole text, and rounds 5 to 12 reviewed deltas. Distinct blockers by round: 13, 10, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0. Every blocker was accepted and corrected, none rebutted. The mechanism of the dirty mark was replaced once, in round 2, after reviewers found stale results returned as current in several orders of events. Rounds 9 to 11 reviewed the changes for the requester's decisions of 2026-11-14 and found major defects in the large sort and in the whole-column rule. All of them were corrected, and round 12 found none.
  • Budget: 6 review invocations were set at the start and raised to 21 in nine steps, each with a stated reason. The reason was the same each time: the blockers fell each round, and the protocol requires an assessment of every change after approval.
  • Final verdicts, quoted from the reports: reviewer A "VERDICT: ACCEPT" on the snapshot that this PR contains. Reviewer B "VERDICT: ACCEPT" on snapshot 10. Reviewer B did not read the last two deltas. Those deltas apply the corrections that reviewers B and A listed for snapshots 10 and 11, and reviewer A assessed them.
  • Follow-ups that stay open, all minor, each closed by one sentence, and none needs a new mechanism:
    • A stream of edits that keeps more than 8 files of the sorted range dirty can end a large sort as unavailable after 10 minutes with no content changed. Spec rule 41 and the Large sort criterion do not say so.
    • Settle waits for a failed time record, so an export that did not ask for a look can wait up to 10 minutes and then fail (technical design 13.4 and 8.4).
    • A Public read shows time formulas as pending while any time record is open, because its budget is 0 (technical design 8.4, spec rules 23 and 77).
    • While a time record is open, a time formula and a formula that reads it can show different times on one screen.
    • Some sentences use a colon as a connector, which flats-unslop discourages.
  • The reports and the task report are local to the author's machine and are not attached. This description is their accessible record.

Checks

make docs-check, make docs-audit, the documentation tests (./docs/..., ./plugins/..., ./internal/mcpx/..., ./internal/doccheck/... and the two runtime documentation tests), compiling every package, and go vet ./... all pass on this tree. make is not installed on the author's Windows machine, so the Makefile commands ran directly.

Add the product spec and the technical design of the sheet content type: a
workbook of worksheets that people and agents edit live in a cell grid, with
formulas that follow Excel and xlsx and CSV import and export. Both pages have
the status planned. The change adds no code, no guide topic and no tool.

The design answers the ten design decisions of issue 233. It rests on the S0
prototype reports and on small probes that measured the dirty mark, the xlsx
reader and writer, and find in the bundled SQLite. Reviewers of other models
reviewed the text in eight rounds and accepted it. One of them assessed the last
delta.

Refs #233
…sign

The requester decided the open questions of the sheet spec on 2026-11-14 and
changed two of them.

A sort of 100,001 to 2,000,000 cells is now a large sort: a job that plans
while people keep editing and applies in one writer item. A digest of the sort
keys, the structure sequence and the merged ranges guard the apply, and a
failed guard changes no content. The `ids` record of the apply names the
columns of the range, so `refs` gets a second index on the source column.

A whole-column reference now covers the whole grid, as in Excel. A function
that reads it directly costs only the rows that exist. An array expression
over it counts every position toward the work limit, and the array limit is
4,194,304 values.

The spec lists questions 3 to 17 as decided. Reviewers of other models
reviewed the change in four more rounds and accepted it.

Refs #233
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant